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Introduction 
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There are a number of technological challenges and opportunities awaiting 
librarians, whether they are cognizant of them or not. Questions abound about the role 
librarians will play within the scope of the Internet and the World Wide Web, questions 
which will unfold as the electronic age evolves. 

The problem 

This research paper applies an evaluation method for World Wide Web sites 
that provide access to online reference materials at academic and public libraries. 
Scrutinizing reference sites with a set of criteria may further the development of better 
library Web sites. It is hoped that librarians will come to understand the need for an 
evaluative process of the World Wide Web and other multimedia in order to maintain 
relevancy for current patrons and promote usage for potential patrons. 

The objectives 

To identify criteria or standards for the evaluation of Web pages in a library 
reference departments. 

To compose a set of criteria for the scrutiny of Web pages in library reference 
departments as well as Web pages created by commercial efforts. 

To compare and contrast the Web pages analyzed based on a set of criteria. 

To consider the possible connection between Web page quality and library size 
and type. 
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V 
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Assumptions and Limitations 

This paper assumes that libraries of all shapes and sizes are actively pursuing 
a presence on the World Wide Web, and specifically reference departments. It 
assumes some reference librarians are evaluating Web sites as part of their job 
descriptions. 

Most of the sources cited in the literature review were published in print, 
although the nature of the Internet enables easy electronic publishing. However, since 
electronic journals have not been placed under the same quality rigors as print 
publications, such as peer and/or editorial review, print publications have been cited 
more often in this paper. Ideally, most of the sources for such a paper about the Web 
should come from the Internet and not from print sources. 

This paper does not assume to be an exhaustive analysis of all reference 
department Web sites. More importantly, it should be noted the evaluation of Web 
pages is very much a subjective process and is not intended to defame or discredit 
any institution. As long as people are involved with the criticism of Web pages, this 
process will be imperfect and open to debate. 
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Methodology 

The evaluation of Web sites was performed using a model developed from 
sources mentioned in the literature review. The model is in a questionnaire form, 
which follows: 

I. Web site currency, accuracy and relevancy 

1 . Did the hypertext links retrieve the site or was the URL not found: on a 

scale of 0 to 10, with 0 being all links inaccessible, and 10 being 
all links accessible. 

2. When was the last time the site was updated? 

3. Are the hypertext links appropriate and relevant for an online reference 

desk or are the sites of questionable value; on a scale of 0 to 1 0, 
with 0 being the least relevant and appropriate links and 10 being 
the most relevant and appropriate links. 

II. Web site organization / structure 

1 . Is the site accessible from Netscape 2.0 browsers: with yes answers 

receiving a 10 and no answers receiving a 0. 

2. Do all images, icons and graphics paint when the Web page loads; on 

a scale of 0 to 10, with 0 being none of the images, icons and 
graphics loading and 10 being all of the images, icons and 
graphics loading. 

3. How fast does the Web page paint; on a scale of 0 to 10, with 0 being 

the slowest and 10 being the quickest. 




III. Web site presentation: 
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1 . Are any graphics and text blurry, difficult to read or comprehend; on a 

scale of 0 to 10, with 0 being the most unreadable, difficult to read 
or comprehend and 10 being the most clear, easy to read and 
comprehend. 

2. Are there too many or too few graphics; on a scale of 0 to 10, with 0 

being too many or too few graphics and 10 being the most 
appropriate middle ground for graphics use. 

3. Are the Web pages easy to navigate; on a scale of 0 to 10, with 0 being 

the most difficult to navigate and 10 being the easiest to navigate. 

IV. URL maintenance 

1 . Are the names of the people or persons responsible for the site 

displayed; with yes answers receiving a 1 0 and no answers 
receiving a 0. 

2. Is there a <mailto> HTML tag which allows a user to send feedback or 

comments to the library; with yes answers receiving a 10 and no 
answers receiving a 0. 

3. Is there a disclaimer about the validity or reliability (or lack thereof) of 

the information provided by the institution; with yes answers 
receiving a 1 0 and no answers receiving a 0. 

V. Web site features 

1. Is there an option to search a library’s OPAC or online databases; with 

yes answers receiving a 10 and no answers receiving a 0.? 

2. Is there an option to request reference assistance (from a human 

being) such as via email, online chatting or virtual reality; with yes 
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answers receiving a 1 0 and no answers receiving a 0. 

3. Are there hypertext links to other Web reference sites; with yes 

answers receiving a 1 0 and no answers receiving a 0. 

4. Are there hypertext links to the community or region; with yes answers 

receiving a 10 and no answers receiving a 0. 

5. Is there a search engine of the reference sites made available; with 

yes answers receiving a 1 0 and no answers receiving a 0. 

Average scores were tabulated from each of the five evaluation categories. 
Those averages were added together. Then, the Web sites were compared and 
contrasted based on those scores. The reviewer’s subjective and objective comments 
about the Web sites were used to discuss the evaluations based in part on a loose set 
of criteria mentioned in the literature review. 

The six sites were chosen after scanning numerous web sites at Berkeley 
Digital Library SunSITE’s “Libraries On The Web" in the USA 
(http;//sunsite. Berkeley. EDU/Libw\eb/usa.html). The Berkeley site lists both public and 
academic libraries, regardless of whether they contain an online reference site or not. 
The sites were chosen after meeting two criteria which can be determined from the 
1 995 edition of the American Library Directory, which include; 

1 . ) Is the library automated? 

2. ) Does the library contain at least 100,000 book titles? 

Also, library reference web sites were chosen if they; (1.) provided links to web 
sites which can be used to answer reference questions for both librarians and/or 
patrons; (2.) were developed uniquely by the specific library and not merely a link to 
another library’s reference web site; (3.) the library included some indication of 
responsibility or accountability for the web site with a email address of a librarian’s 
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name, web design team name or library name; (4.) the web site indicated when it was 
last updated. 

Sixty web sites were viewed from the “Libraries On The Web” list, with half 
being public libraries and the other half academic libraries. It should be noted that 
roughly 75 percent of the sites had lists of links to answer reference questions, 
although some of these sites were called Internet resources, subject links or favorite 
web sites. Except for the New Orleans Public Library, all of the sites evaluated here 
use the term “reference” or “ready reference” to describe their web resource links. 
Three reference web sites from university libraries and three reference web sites from 
public libraries were selected from the “Libraries On The Web" Web page that met the 
above-mentioned criteria. 
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Hypothesis 

The evaluations were expected to demonstrate a difference between those sites 
which have been updated more recently than those that have not. Also, it was 
anticipated that academic libraries would make better use of graphics than public 
libraries but little difference would be found in the quality of the links to Internet 
reference resources. 
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Definitions * 

Browser 

A client program (software) that is used to looking at various kinds of Internet 
resources. 

Cyberspace 

Term originated by author William Gibson in his novel "Neuromancer", the word 
Cyberspace is currently used to describe the whole range of information 
resources available through computer networks. 



E-mail 

(Electronic Mail) -- Messages, usually text, sent from one person to another via 
computer. E-mail can also be sent automatically to a large number of addresses 
(Mailing List). 



HTML 

(Hypertext Markup Language) -- The coding language used to create Hypertext 
documents for use on the World Wide Web. HTML looks a lot like old-fashioned 
typesetting code, where you surround a block of text with codes that indicate 
how it should appear, additionally, in HTML you can specify that a block of text, 
or a word, is "linked " to another file on the Internet. HTML files are meant to be 
viewed using a World Wide Web Client Program, such as Mosaic. 

HTTP 

(Hypertext Transport Protocol) -- The protocol for moving hypertext files across 
the Internet. Requires a HTTP client program on one end, and an HTTP server 
program on the other end. HTTP is the most important protocol used in the 
World Wide Web (WWW). 

Hypertext 

Generally, any text that contains "links" to other documents -words or phrases in 
the document that can be chosen by a reader and which cause another 
document to be retrieved and displayed. 

Internet 

(Upper case I) The vast collection of inter-connected networks that all use the 
TCP/IP protocols and that evolved from the ARPANET of the late 60's and early 
70's. The Internet now (July 1995) connects roughly 60,000 independent 
networks into a vast global internet. 

internet 

(Lower case i) Any time you connect 2 or more networks together, you have an 

* Definitions from Matisse Enzer and Internet Literacy Consultants . 
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internet - as in inter-national or inter-state. 

Listserv 

The most common kind of maillist, Listservs originated on BITNET but they are 
now common on the Internet. 

Netscape 

A WWW Browser and the name of a company. The Netscape (tm) browser was 
originally based on the Mosaic program developed at the National Center for 
Supercomputing Applications (NCSA). Netscape has grown in features rapidly 
and is widely recognized as the best and most popular web browser. Netscape 
corporation also produces web server software. Netscape provided major 
improvements in speed and interface over other browsers, and has also 
engendered debate by creating new elements for the HTML language used by 
Web pages - but the Netscape "extensions" to HTML are not universally 
supported. The main author of Netscape, Mark Andreessen, was hired away 
from the NCSA by Jim Clark, and they founded a company called Mosaic 
Communications and soon changed the name to Netscape Communications 
Corporation. 



URL 

(Uniform Resource Locator) - The standard way to give the address of any 
resource on the Internet that is part of the World Wide Web (WWW). A URL looks 
like this: 

http://www.matisse.net/seminars.html 
or telnet://well.sf.ca.us 
or news:new.newusers.questions 

The most common way to use a URL is to enter into a WWW browser program, 
such as Netscape, or Lynx. 

Usenet 

A world-wide system of discussion groups, with comments passed among 
hundreds of thousands of machines. Not all Usenet machines are on the 
Internet, maybe half. Usenet is completely decentralized, with over 10,000 
discussion areas, called newsgroups. 

WWW or Web 

(World Wide Web) - Two meanings - First, loosely used: the whole constellation 
of resources that can be accessed using Gopher, FTP, HTTP, telnet, Usenet, 
WAIS and some other tools. Second, the universe of hypertext servers (HTTP 
servers) which are the servers that allow text, graphics, sound files, etc. to be 
mixed together. 
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II. Literature Review 
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Librarians' Roles in the Future 

When a librarian asks about his or her place in the emerging world of online 
media, such as the World Wide Web, he or she probably wonders where to begin. 
There are so many separate and distinct parts to know about the Internet. This 
literature review reflects the changing roles of librarians and information scientists; the 
composition of the virtual library and how librarians will perform within that 
environment; and a sporadic number of articles which consider users needs in the 
virtual library. There is scant literature about web standards and criteria, although this 
is exactly what is needed if librarians are to understand and better serve patrons via 
electronic resources. No discussion has been found in this literature review 
questioning whether libraries should participate with technologies such as the World 
Wide Web -- the question now is how can librarians serve patrons as effectively in 
cyberspace as they do in the real world. 

Just how often do reference librarians utilize the Internet? This question was the 
basis of a survey by Geetali Basu in 1994, who found that “many librarians were not 
using [sic] Internet to answer' reference questions" (Basu 38). 

Librarians must decide what role they will play within the new technology of the 
World Wide Web, as well as future environments. Numerous articles discuss the 
potential roles of librarians. One basic issue is whether to mediate on the behalf of 
patrons or not. “Should librarians offer ‘reference services’ which are aimed at taming 
the wild frontier that is currently the Internet? Or perhaps for some the question is, can 
librarians offer such services?” (Summerhill 142). This question goes back to the root 
differences between conservative and liberal librarianship -- serve many patrons at a 
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basic level while educating them or serve fewer patrons with advanced service at the 
expense of others. There may be some patrons reluctant or unwilling to venture into 
the Internet; yet library staff time and money, already strained in this fiscally 
conservative time in some cases, may be tunneled in that direction. 

Consultant is another role for librarians to consider in the Information Age. One 
viewpoints believes librarians must assist users to navigate through the glut of data 
and information available on the Internet (Myers 637). Another view is that librarians 
must not be “intermediaries,” since “[electronic libraries are being built along similar 
self-service lines. They are designed for use with librarians serving not as 
intermediaries but as consultants” (Barnes 58). Again, this discussion boils down to 
the conservative versus liberal debate in librarianship. 

New technology requires librarians to acquire a new set of skills. One study at 
Cornell University’s Mann Library discovered the library was serving roughly the same 
number of patrons as they had ten years previously, but had a 50 percent increase in 
reference questions over the same time (Barnes 49). “More databases for users seem 
to bring more questions from users -- questions about database content, database 
selection, and system use ..." (Barnes 49) Technological advancements have made 
more information available more frequently. Users hope they can find assistance from 
anyone willing to provide information in the format they need without being “... buried 
in piles of information” (McClure 54). 

Collecting, organizing and cataloging data from the Internet and other electronic 
media, such as CD-ROM technology, will increase greatly; 

Everybody will need skills to retrieve and manage information to 
get along. Many will probably find the large quantity of information 
overwhelming, and will seek education or assistance in improving their 
information retrieval and management skills (Desmarais 164). 

Until recently, when World Wide Web search engines became plentiful, the Internet 
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was a means to make lots of data available world-wide with the assumption the 
someone would arrange it (McClure 54). 

Librarians will be asked to serve as cyber-lifeguards on the virtual shoreline of 
the Internet, helping users or “surfers” stay afloat. In other words, librarians will aid 
users make sense of what they have found or provide better search techniques. States 
Charles Lowry: “It is still perilously easy for ordinary users to become lost on the 
Internet, leading to frustration and decreasing the likelihood that they will discover 
something useful” (49). Evaluating Web sites and other Internet tools will become a 
part of this new reference service patrons will expect and is another avenue for 
librarians to pursue in order not to become defunct, just as bank tellers are today since 
the invention and acceptance of ATM machines. “Internet sources ... should be subject 
to the same rigorous attention given paper and CD-ROM sources: the content, 
usability, and cost should be evaluated and compared with other sources" (Falcigno 
24). Not only should librarians be trained on how to operate within the Internet and 
electronic environments, but they must be educated on how to best utilize them (Bruce 
39). It would be simple to provide links to various Web sites without regard to their 
content; thus providing access, but what about content? 

Librarians will have to evaluate web resources to assure the 
quality of the information provided. It is fair to say that this will be one 
of the big problems with web resources, and one for which there are 
no easy answers” (Healey 444). 

One controversial task some librarians envision is that of a communal network 
for reference questions. Obviously, this view is supported by the conservative / 
minimalist camp, who dislike answering basic questions repeatedly (Myers 636). They 
see a chance to provide basic information in FAQ format (frequently asked questions), 
a term for answers to questions frequently asked by new Internet users. 

Myers and Humphreys hope librarians around the world begin to collaborate 





13 



their knowledge just like catalogers who incorporate Library of Congress’ MARC 
records (machine-readable records) into their online public access catalog (OPAC). 
(636) (460). Automating the reference desk would involve cataloging reference 
questions and answers as hyperlinks to the reference collection via the OPAC and 
making it available to other reference librarians (Humphreys 460-2). Although this is a 
radical idea, part of this notion already exists. The Internet listserv called “STUMPERS- 
L” provides librarians, patrons, students and researchers from around the world the 
ability to ask difficult reference questions in the hope someone will know the answer 
(STUMPERS). 

Lowry foresees a “new library paradigm” that will occur because of the 
integration of the Internet into the library “technology environment” as well as the 
expert assistance from librarians, who will “guide library patrons toward resources that 
are useful” (48). There are ample opportunities for librarians to continue serving their 
patrons, but librarians must decide just how much time, money and effort they will 
expend for users in the virtual library. 
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Users' needs 

In the telecommuting age, we still have the need to know people 
as people, not just as words on a flickering screen (Reid 221). 

Users should be the prime concern for librarians. Obviously, librarians would 
not have careers without users visiting the library, either physically or electronically. 
Librarians who are responsible for designing and maintaining web sites should 
consider making the web site as easy to use as possible and available to the lowest 
common denominator in the technological sense. 

Aside from the above mentioned layout considerations, there is a need for 
librarians to understand and appreciate users’ experiences with the library. Although 
many librarians can pursue a reference question doggedly, delving into various 
sources to come up with the right answer if they choose, it would seem users are not 
as patient. “What is a user’s information tolerance?” LaGuardia states, adding: 

Just how much information overload can you stand? What 
is your technostress tolerance? .... How do you feel about wading 
through four hours of online dreck [sic] before finding a kernel of 
research truth? (43). 

As technology advances, librarians who decide to not keep pace with the rest of 
society may find themselves irrelevant to their more educated patrons. One example of 
increasing patron demands, based on technology, is the delivery of information. 

Before online commercial vendors were able to provide full-text magazine articles, 
library patrons had to wait at least several weeks in order to receive materials through 
their library’s interlibrary loan department if a library did not subscribe to a particular 
journal. Now, with full-text resources instantaneously available, users will deem any 
service slower than that benchmark as unacceptable (Summerhill 139). Whether this 
expectation is justified or not is irrelevant, since its the users who determine whether 
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libraries are worth their time. 

Since users’ needs and demands are changing, librarians will be forced to 
adapt their delivery and access to information. Originally, librarians were just 
concerned about automating their libraries’ basic functions and card catalog. But now 
that librarians really are not performing mediated online searches as they did twenty 
years ago, the ease of use for users’ has become a prime focal point for access (Lowry 
50). 

Now, the vision is of client software running on the users desktop 
computer and retrieving bibliographic, full-text, and (eventually) 
multimedia information stored on servers located at various sites. The 
user interface of the virtual library client must enable users to easily navigate 
an increasingly complex information environment and manipulate the 
information retrieved. ... To understand how we will provide access in 
the future, we must think in terms of the common user interface, not the 
OPAC. To crib a slogan, “It’s the GUI, stupid!” (Lowry 50). 

Either librarians consider how they will deliver information within the framework 
of the technological future and an eye focused on users’ expectations and needs, or 
they will become irrelevant to users. 
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Virtual, Digital and Electronic libraries 

Discussion of librarians’ future role in the Information Age can not be accurately 
depicted without an examination of the virtual library. Specifically, how will reference 
librarians operate within this new library setting? Many articles in the literature review 
apparently apply the terms virtual, electronic and digital synonymously, except for 
Reid’s article, which will be discussed later. 

A paradigm shift is occurring as librarians consider their place in the virtual 
library, determined in part by librarians’ openness to new methods of service: 

Our response to this change (and the opportunities it presents) 
ought to be a willingness to experiment vigorously with new services and 
technologies, resulting in the selection of those that hold the greatest 
promise of providing information resources for the future. ... We must 
formulate a vision that will result in a foundation for the virtual library that 
will work at least as well as the old paradigm worked in its time (Lowry 39). 

It seems inevitable that reference librarians will be answering questions via 
email, which will leave libraries open to worldwide service (McClure 54). In order to 
deal with great demand for email reference service, librarians may set up an 
automated FAQ center or “hypertext search guide” (Myers 636). Users within the Mann 
Library at Cornell University can request a librarian to assist them by sending an 
electronic message from their terminal (Barnes 49). There will be new ways for users 
to interact with librarians, but not necessarily better ways. 

As an “electronic scholarly communications system,” the WWW offers many 
advantages to users, including “ ‘ease of use, standardized language, extremely 
inexpensive, hypertext format [to] enable browsing and serendipity, customizable, 
always available, expandable and is accepted’ ” (Gorka 21). Other advantages to 
libraries include their ability to operate with various aspects of the Internet, including 
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gophers, WAIS, ftp and telnet sites; web servers can easily establish a library’s 
presence on the Web; an abundance of information on any topic; and integrating 
various sites into one central location enables precise or expanded access to Web 
sites (Healey 442). 

Librarians will be challenged by users and other librarians to make sense of this 
overabundance of information. To that end, librarians have begun to catalog Internet 
sites in the hope of integrating virtual resources within the OPAC. One study attempted 
to apply MARC cataloging rules to FTP sites (Dillon 88, 90-1). Client-server technology 
is moving the library into cyberspace. “Client-server architecture is the key concept 
underlying mass distribution via multiple channels. Servers make all kinds of 
information available to computers wired to a network. Clients provide an interface to 
the information offered up from servers” (Alberico 31). 
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Balance between Print and Electronic media 

Since the nature of information will increasingly involve electronic multimedia 
and full-text, libraries should be prepared to provide access to these electronic media, 
just as libraries have provided access to books. It will require a commitment on the part 
of “digital research libraries" to accept electronic information in this fashion (Graham 
332). These select few digital research libraries will become regional warehouses for 
electronic information and must be “committed to organizing, storing and providing 
electronic information for periods of time longer than human lives." (332). 

Collection development policies would be needed in order to determine which 
information is most useful for storage, considering costs will also prevent these digital 
research libraries from collecting all electronic data (Graham 333). 

In the practical sense, collection development policies will be necessary in 
order to avoid overwhelming users visiting a library’s WWW site. This could cause 
“technostress” to patrons. “[Ajdding too many links leaves the user feeling as if they 
could do better on their own” (Falcigno 26). The nature of the Web requires librarians 
to change their view of the collection development process as passive. “... [Electronic 
information is dynamic. To use it, one must interact with the store of information, 
spread across a loose network and constantly changing” (Alberico 31). 

Initially, if a library was to decide to archive electronic media, it must believe that 
electronic media is as important as print sources (Graham 337). This 
acknowledgement does not mean libraries will treat information in either format as 
having the same properties because, in fact, they do not. 

Print media has a remarkable resiliency and simplicity that electronic reference 
resources can only imagine. “The book is a marvelous technology for telling extended 
stories and making overarching statements. Books will be around for a long time” 
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(Myers 638). Although the Internet provides access to numerous amounts of data, 
scattered worldwide, it currently can not approach the convenience and authority of 
print sources in answering reference questions. “Simple data retrieval ... can be 
retrieved at present more efficiently and quickly by reaching for a standard printed 
reference tool” than Web site (Santa Vicca 233). 

The key to the debate between print and electronic sources is to utilize the best 
aspects of each format so librarians can provide good reference service to patrons 
(LaGuardia 43). 
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Technological Difficulties 

Technological advancement does not come without a price, and not just a 
monetary price. Hardware and software must compliment each other correctly in order 
to access information found in electronic format, thus resulting in a “co-dependent" 
relationship between librarians and users -- and technology (Cady 207). “With the 
cancellation of paper copies in favor of CD-ROM products, the malfunctioning of any of 
the complicated parts and pieces of our CD-ROM network ... means that virtual library 
patrons are locked out of liberspace as completely as if we closed the physical 
buildings at random during the day” (207). 

More specifically, data and information found on the Internet lack some of the 
qualities librarians and patrons take for granted. Some of these limitations include “ ‘a 
lack of consistent classification scheme or graphical cues to aid users; authority control 
is absent or difficult to determine; data and information are rarely evaluated for content; 
software does not filter the data by subject or content; and keyword-Boolean operators 
are inadequate for locating full-text resources’ ” (Lowry 48). For reference librarians, 
the lack of indexing on the Internet really inhibits its usefulness as a reference tool 
(Summerhill 137). Other problems include the volatility of Web sites and email 
addresses, which means important information or service may suddenly move without 
notice. Also, Internet traffic can become clogged at various times of the day (Bruce 40). 
“Likewise evaluating resources becomes a problem if the resource cannot be located” 
(Bruce 40). 

Other problems for librarians attempting to use the Internet as a reference tool 
include “the ability to connect ... the searcher’s level of expertise ... the ability to quickly 
retrace your steps” among other problems mentioned above (Basu 39). Simply 
conducting a reference interview through the telephone is difficult enough -- imagine 
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attempting to understand someone’s query sent via email. “How can you relate to 
someone you do not see?” (Reid 217). Also, the long search time required for Internet 
reference questions and the investment in librarian training will continue to hold back 
the day-to-day implementation of the Internet into reference work (Augustine 22). 
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Web Standards, Criteria & Design 

There are a few important facets to information provided through the WWW, 
qualities that users currently expect and demand as well as librarians who must 
initially “surf” the Internet to find information for their patrons. There are only a few 
sources of criteria that have been published in scholarly journals or exist on the Web 
and there is a definite need for more research and study in this area. Based on the 
standards collected, researchers tend to group criteria as lists with brief annotation. 
Although the criteria are generally similar, it is the combination of criteria that sets one 
standard from another. 

Convenience should be a prime consideration for web designers, which can be 
obtained by selecting sites that have full-text documents rather than just bibliographic 
citations. “The more the resource provides the actual information needed ... the more 
students like it” (Cady 208-9). 

Quality is another consideration, closely associated with convenience. 

Quality as a goal for Web information involves a continuous 
process of planning, analysis, design, implementation, and development 
to ensure that the information meets users needs in terms of both 
content and interface (December 1119). 

Criteria for web quality revolves around information through verifying that 
information is: “ ‘correct,’ ‘accessible’, ‘usable’, ‘understandable’, and ‘meaningful’ " 
(December 1 1 1 9-20). Similar web page standards have been adapted from the efforts 
of Jakob Neilsen which include: “ ’reliability, distinctiveness, easy to use/remember, 
accuracy, modularity [an interdependent unit structure of the Web site which enables 
adding pages easily], optimization [a Web site designed for a variety of formats and 
browsers], and a [mechanism for] user feedback’ ” (McCleod 48). Another set of 
criteria involve “purpose, authority, scope, audience, cost, format, special features and 
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the existence of reviews which might speak to any or all of the criteria above” (Katz 23- 
29). 



Obviously, any set of criteria combinations is possible and is as subjective an 
assessment as the actual evaluation or review of a web site. Another list of criteria 
includes the following criteria: 

ease of finding out the scope and criteria for inclusion that lets the evaluator see 
if it matches a library’s needs 

ease of identifying: the authority of authors, the currency, the last update, 
and what was updated. 

stability of information 

ease of use:convenience or organization and speed of connection (Tillman) 

In a related issue, there is also a need for Internet site selection standards, 
similar to a collection development policy for a library’s Web pages. One such article 
discusses several possible factors for Internet site selection, including the “ ‘quality 
and content of the source, relevancy, ease of use, reliability and stability, cost and 
copyright issues, and hardware and software compatibility’ ” (Pratt 134-5). 

A collection of roughly 100 university Web publishing guidelines is a useful tool 
for libraries looking to devise their own polices. (Kimmel) 

Yale University’s manual describes the design principles used to create the 
pages within the Center for Advanced Instructional Media's (C/AIM) World Wide Web 
site. The manual’s section on World Wide Web site structure states: 



The user's perception of and assumptions about the organization 
of your World Wide Web (WWW) site can have a major impact on the 
usability of your page and site design. The power of WWW lies in the 
myriad of links that are possible with hypertext linkage of documents and 
Web sites. ... Good graphic design always seeks the optimal balance 
between visual sensation and graphic and text information (Lynch). 
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Evaluation Form for Mansfield University 

http://www.mnsfld.edu/depts/lib/index.html 

I. Web site currency, accuracy and relevancy 

1 . Did the hypertext links retrieve the site or was the URL not found; on a scale 

of 0 to 10, with 0 being all links inaccessible, and 10 being all links 
accessible. 1 0 

2. When was the last time the site was updated? Mav 21. 1996 

3. Are the hypertext links appropriate and relevant for an online reference desk 

or are the sites of questionable value; on a scale of 0 to 10, with 0 being 
the least relevant and appropriate links and 10 being the most relevant 
and appropriate links.2. 

II. Web site organization / structure 

1 . Is the site accessible from Netscape 2.0 browsers; with yes answers 

receiving a 10 and no answers receiving a 0.1 0 

2. Do all images, icons and graphics paint when the Web page loads; on a 

scale of 0 to 10, with 0 being none of the images, icons and graphics 
loading and 10 being all of the images, icons and graphics loading. 1JL 

3. How fast does the Web page paint; on a scale of 0 to 10, with 0 being the 

slowest and 10 being the auickest.1 0 

III. Web site presentation: 

1 . Are any graphics and text blurry, difficult to read or comprehend; on a scale of 

0 to 10, with 0 being the most unreadable, difficult to read or comprehend 
and 10 being the most clear, easy to read and comprehend.! 

2. Are there too many or too few graphics; on a scale of 0 to 1 0, with 0 being too 

many or too few graphics and 10 being the most appropriate middle 
ground for graphics use.4 

3. Are the Web pages easy to navigate; on a scale of 0 to 10, with 0 being the 

most difficult to navigate and 1 0 being the easiest to navigate.5. 

IV. URL maintenance 

1 . Are the names of the people or persons responsible for the site displayed; 

with yes answers receiving a 1 0 and no answers receiving a 0.1 0 

2. Is there a <mailto> HTML tag which allows a user to send feedback or 

comments to the library; with yes answers receiving a 10 and no answers 
receiving a 0. IQ. 

3. Is there a disclaimer about the validity or reliability (or lack thereof) of the 

information provided by the institution; with yes answers receiving a 10 
and no answers receiving a 0. Q. 

V. Web site features 

1. Is there an option to search a library’s OPAC or online databases; with yes 
answers receiving a 1 0 and no answers receiving a 0. 1_Q_ 
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2. Is there an option to request reference assistance (from a human being) such 

as via email, online chatting or virtual reality; with yes answers receiving 
a 1 0 and no answers receiving a O.Q, 

3. Are there hypertext links to other Web reference sites; with yes answers 

receiving a 10 and no answers receiving a 0. 1 0 

4. Are there hypertext links to the community or region; with yes answers 

receiving a 10 and no answers receiving a 0.1 0 

5. Is there a search engine of the reference sites made available; with yes 

answers receiving a 10 and no answers receiving a 0.Q, 



Total Score = 37.4 
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Evaluation Form for Morton Grove Public Library 

http://www.nslsilus. 0 rg/mgkh 0 me/ 0 rrs/ 0 rrsmenu.html#t 0 p 

I. Web site currency, accuracy and relevancy 

1 . Did the hypertext links retrieve the site or was the URL not found: on a scale 

of 0 to 10, with 0 being all links inaccessible, and 10 being all links 
accessible. 1 0 

2. When was the last time the site was updated? Mav 13. 1996 

3. Are the hypertext links appropriate and relevant for an online reference desk 

or are the sites of questionable value: on a scale of 0 to 1 0, with 0 being 
the least relevant and appropriate links and 10 being the most relevant 
and appropriate links.g. 

II. Web site organization / structure 

1. Is the site accessible from Netscape 2.0 browsers: with yes answers 

receiving a 10 and no answers receiving a 0.JJL 

2. Do all images, icons and graphics paint when the Web page loads; on a 

scale of 0 to 10, with 0 being none of the images, icons and graphics 
loading and 10 being all of the images, icons and/graphics loading.^. 

3. How fast does the Web page paint: on a scale of 0 to 1 0, with 0 being the 

slowest and 10 being the quickest. 1 0 

III. Web site presentation: 

1 . Are any graphics and text blurry, difficult to read or comprehend: on a scale of 

0 to 10, with 0 being the most unreadable, difficult to read or comprehend 
and 10 being the most clear, easy to read and comprehend.^ 

2. Are there too many or too few graphics: on a scale of 0 to 10, with 0 being too 

many or too few graphics and 1 0 being the most appropriate middle 
ground for graphics use.fi. 

3. Are the Web pages easy to navigate: on a scale of 0 to 10, with 0 being the 

most difficult to navigate and 10 being the easiest to navigate.^. 

IV. URL maintenance 

1 . Are the names of the people or persons responsible for the site displayed: 

with yes answers receiving a 10 and no answers receiving a 01_Q. 

2. Is there a <mailto> HTML tag which allows a user to send feedback or 

comments to the library: with yes answers receiving a 1 0 and no answers 
receiving a 0. 1SL 

3. Is there a disclaimer about the validity or reliability (or lack thereof) of the 

information provided by the institution: with yes answers receiving a 10 
and no answers receiving a 0. 1_0. 

V. Web site features 

1 . Is there an option to search a library’s OPAC or online databases: with yes 
answers receiving a 1 0 and no answers receiving a 0. 1 0 
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2. Is there an option to request reference assistance (from a human being) such 

as via email, online chatting or virtual reality; with yes answers receiving 
a 10 and no answers receiving a O.Q. 

3. Are there hypertext links to other Web reference sites; with yes answers 

receiving a 10 and no answers receiving a 0.0 

4. Are there hypertext links to the community or region; with yes answers 

receiving a 10 and no answers receiving a 0 1 0 

5. Is there a search engine of the reference sites made available; with yes 

answers receiving a 10 and no answers receiving a 0.Q, 



Total Score = 41.1 
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Evaluation Form for New Orleans Public Library 
http://www.gnofn.org/~nopl/ 

I. Web site currency, accuracy and relevancy 

1 . Did the hypertext links retrieve the site or was the URL not found; on a scale 

of 0 to 10, with 0 being all links inaccessible, and 10 being all links 
accessible.8 

2. When was the last time the site was updated? Mav 15. 1996 

3. Are the hypertext links appropriate and relevant for an online reference desk 

or are the sites of questionable value; on a scale of 0 to 1 0, with 0 being 
the least relevant and appropriate links and 1 0 being the most relevant 
and appropriate links, g. 

II. Web site organization / structure 

1. Is the site accessible from Netscape 2.0 browsers; with yes answers 

receiving a 10 and no answers receiving a 0.1 0 

2. Do all images, icons and graphics paint when the Web page loads; on a 

scale of 0 to 10, with 0 being none of the images, icons and graphics 
loading and 10 being all of the images, icons and graphics loading.^. 

3. How fast does the Web page paint; on a scale of 0 to 10, with 0 being the 

slowest and 1 0 being the quickest.fi. 

III. Web site presentation: 

1 . Are any graphics and text blurry, difficult to read or comprehend; on a scale of 

0 to 10, with 0 being the most unreadable, difficult to read or comprehend 
and 10 being the most clear, easy to read and comprehend.^. 

2. Are there too many or too few graphics; on a scale of 0 to 1 0, with 0 being too 

many or too few graphics and 1 0 being the most appropriate middle 
ground for graphics use.2 

3. Are the Web pages easy to navigate; on a scale of 0 to 1 0, with 0 being the 

most difficult to navigate and 10 being the easiest to navigate. 1 0 

IV. URL maintenance 

1 . Are the names of the people or persons responsible for the site displayed; 

with yes answers receiving a 10 and no answers receiving a 0.1 0 

2. Is there a <mailto> HTML tag which allows a user to send feedback or 

comments to the library; with yes answers receiving a 10 and no answers 
receiving a 0. IQ. 

3. Is there a disclaimer about the validity or reliability (or lack thereof) of the 

information provided by the institution; with yes answers receiving a 1 0 
and no answers receiving a O.Q. 

V. Web site features 

1 . Is there an option to search a library’s OPAC or online databases; with yes 
answers receiving a 1 0 and no answers receiving a 0. 1 0 
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2. Is there an option to request reference assistance (from a human being) such 

as via email, online chatting or virtual reality; with yes answers receiving 
a 1 0 and no answers receiving a O.Q. 

3. Are there hypertext links to other Web reference sites; with yes answers 

receiving a 10 and no answers receiving a O.Q. 

4. Are there hypertext links to the community or region; with yes answers 

receiving a 10 and no answers receiving a 0.1 0 

5. Is there a search engine of the reference sites made available; with yes 

answers receiving a 1 0 and no answers receiving a O.Q. 



Total Score = 33.5 
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Evaluation Form for Purdue University 

http://thorplus.lib.purdue.edU/reference/index.html#periodic 

I. Web site currency, accuracy and relevancy 

1 . Did the hypertext links retrieve the site or was the URL not found; on a scale 

of 0 to 10, with 0 being all links inaccessible, and 10 being all links 
accessible.fi 

2. When was the last time the site was updated? December 18. 1995 

3. Are the hypertext links appropriate and relevant for an online reference desk 

or are the sites of questionable value; on a scale of 0 to 10, with 0 being 
the least relevant and appropriate links and 10 being the most relevant 
and appropriate links, fi 

II. Web site organization / structure 

1 . Is the site accessible from Netscape 2.0 browsers; with yes answers 

receiving a 10 and no answers receiving a 0.1 0 

2. Do all images, icons and graphics paint when the Web page loads; on a 

scale of 0 to 10, with 0 being none of the images, icons and graphics 
loading and 10 being all of the images, icons and graphics loading.fi. 

3. How fast does the Web page paint; on a scale of 0 to 10, with 0 being the 

slowest and 1 0 being the quickest. 1 0 

III. Web site presentation: 

1 . Are any graphics and text blurry, difficult to read or comprehend; on a scale of 

0 to 10, with 0 being the most unreadable, difficult to read or comprehend 
and 10 being the most clear, easy to read and comprehend.fi. 

2. Are there too many or too few graphics; on a scale of 0 to 10, with 0 being too 

many or too few graphics and 10 being the most appropriate middle 
ground for graphics use.fi. 

3. Are the Web pages easy to navigate; on a scale of 0 to 10, with 0 being the 

most difficult to navigate and 10 being the easiest to navigate.fi. 

IV. URL maintenance 

1 . Are the names of the people or persons responsible for the site displayed; 

with yes answers receiving a 10 and no answers receiving a O.fi 

2. Is there a <mailto> HTML tag which allows a user to send feedback or 

comments to the library; with yes answers receiving a 10 and no answers 
receiving a 0. 1 0 

3. Is there a disclaimer about the validity or reliability (or lack thereof) of the 

information provided by the institution; with yes answers receiving a 10 
and no answers receiving a O.fi. 

V. Web site features 

1. Is there an option to search a library’s OPAC or online databases; with yes 
answers receiving a 10 and no answers receiving a 0.1 0 
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2. Is there an option to request reference assistance (from a human being) such 

as via email, online chatting or virtual reality; with yes answers receiving 
a10andnoanswersreceivinga0.fi. 

3. Are there hypertext links to other Web reference sites; with yes answers 

receiving a 10 and no answers receiving a 0. 1 0 

4. Are there hypertext links to the community or region; with yes answers 

receiving a 1 0 and no answers receiving a 0. 1 0 

5. Is there a search engine of the reference sites made available; with yes 

answers receiving a 10 and no answers, receiving a 0. 1 0 



Total Score = 35.9 
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Evaluation Form for Appleton Public Library 

http://www.apl.org/pages/lnfoSources.html 

I. Web site currency, accuracy and relevancy 

1 . Did the hypertext links retrieve the site or was the URL not found; on a scale 

of 0 to 10, with 0 being all links inaccessible, and 10 being all links 
accessible.fi. 

2. When was the last time the site was updated? Mav 23. 1996 

3. Are the hypertext links appropriate and relevant for an online reference desk 

or are the sites of questionable value; on a scale of 0 to 10, with 0 being 
the least relevant and appropriate links and 10 being the most relevant 
and appropriate links.fi. 

II. Web site organization / structure 

1 . Is the site accessible from Netscape 2.0 browsers; with yes answers 

receiving a 10 and no answers receiving a 0.1 0 

2. Do all images, icons and graphics paint when the Web page loads; on a 

scale of 0 to 10, with 0 being none of the images, icons and graphics 
loading and 10 being all of the images, icons and graphics loading. 1 0 

3. How fast does the Web page paint; on a scale of 0 to 10, with 0 being the 

slowest and 10 being the quickest. 1 0 

III. Web site presentation: 

1. Are any graphics and text blurry, difficult to read or comprehend; on a scale of 

0 to 10, with 0 being the most unreadable, difficult to read or comprehend 
and 10 being the most clear, easy to read and comprehend. 1 0 

2. Are there too many or too few graphics; on a scale of 0 to 10, with 0 being too 

many or too few graphics and 10 being the most appropriate middle 
ground for graphics use.l 

3. Are the Web pages easy to navigate; on a scale of 0 to 10, with 0 being the 

most difficult to navigate and 10 being the easiest to navigate.fi. 

IV. URL maintenance 

1 . Are the names of the people or persons responsible for the site displayed; 

with yes answers receiving a 10 and no answers receiving a O.fi 

2. Is there a <mailto> HTML tag which allows a user to send feedback or 

comments to the library; with yes answers receiving a 10 and no answers 
receiving a 0. ill 

3. Is there a disclaimer about the validity or reliability (or lack thereof) of the 

information provided by the institution; with yes answers receiving a 10 
and no answers receiving a 0. 1_fi 

V. Web site features 

1 . Is there an option to search a library’s OPAC or online databases; with yes 
answers receiving a 10 and no answers receiving a 0. 1 0 
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2. Is there an option to request reference assistance (from a human being) such 

as via email, online chatting or virtual reality; with yes answers receiving 
a 10 and no answers receiving a 0.0 

3. Are there hypertext links to other Web reference sites; with yes answers 

receiving a 10 and no answers receiving a 0.0. 

4. Are there hypertext links to the community or region; with yes answers 

receiving a 10 and no answers receiving a 0. 1 0 

5. Is there a search engine of the reference sites made available; with yes 

answers receiving a 10 and no answers receiving a O.Q. 



Total Score = 
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Evaluation Form for University of California at Santa Barbara 
http://www.library.ucsb.edu/subj/resource.html 

I. Web site currency, accuracy and relevancy 

1 . Did the hypertext links retrieve the site or was the URL not found: on a scale 

of 0 to 10, with 0 being all links inaccessible, and 10 being all links 
accessible.fi. 

2. When was the last time the site was updated? April 23. 1996 

3. Are the hypertext links appropriate and relevant for an online reference desk 

or are the sites of questionable value: on a scale of 0 to 10, with 0 being 
the least relevant and appropriate links and 10 being the most relevant 
and appropriate links.fi. 

II. Web site organization / structure 

1. Is the site accessible from Netscape 2.0 browsers; with yes answers 

receiving a 10 and no answers receiving a 0.1 0 

2. Do all images, icons and graphics paint when the Web page loads: on a 

scale of 0 to 1 0, with 0 being none of the images, icons and graphics 
loading and 10 being all of the images, icons and graphics loading.fi 

3. How fast does the Web page paint: on a scale of 0 to 10, with 0 being the 

slowest and 1 0 being the quickest, 1_fi 

III. Web site presentation. 

1 . Are any graphics and text blurry, difficult to read or comprehend; on a scale of 

0 to 10, with 0 being the most unreadable, difficult to read or comprehend 
and 10 being the most clear, easy to read and comprehend.fi 

2. Are there too many or too few graphics; on a scale of 0 to 1 0, with 0 being too 

many or too few graphics and 10 being the most appropriate middle 
ground for graphics use.Z 

3. Are the Web pages easy to navigate; on a scale of 0 to 10, with 0 being the 

most difficult to navigate and 1 0 being the easiest to navigate.fi 

IV. URL maintenance 

1 . Are the names of the people or persons responsible for the site displayed; 

with yes answers receiving a 1 0 and no answers receiving a 0.1 0 

2. Is there a <mailto> HTML tag which allows a user to send feedback or 

comments to the library; with yes answers receiving a 10 and no answers 
receiving a 0. 1_fi 

3. Is there a disclaimer about the validity or reliability (or lack thereof) of the 

information provided by the institution; with yes answers receiving a 10 
and no answers receiving a0.fi, 

V. Web site features 

1 . Is there an option to search a library’s OPAC or online databases; with yes 
answers receiving a 10 and no answers receiving a 0.1 0 
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2. Is there an option to request reference assistance (from a human being) such 

as via email, online chatting or virtual reality; with yes answers receiving 
a 10 and no answers receiving a O.Q. 

3. Are there hypertext links to other Web reference sites; with yes answers 

receiving a 10 and no answers receiving a 0.1 0 

4. Are there hypertext links to the community or region; with yes answers 

receiving a 10 and no answers receiving a O.Q, 

5. Is there a search engine of the reference sites made available; with yes 

answers receiving a 10 and no answers receiving a 0. 1 0 

Total Score = 25JL 



Findings 
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Mansfield University’s Web reference site splits its online reference sources into 
two sections, with a list of 1 6 reference subject headings (such as Business and 
Economics, News and Current Events) and a ready reference section. However, there 
is also a section from the library’s menu screen for a third section called the “MU 
Virtual Subject Library” which closely resembles the reference page section 
mentioned above. It appears the MU Virtual Subject Library is being replaced by the 
reference page listing, but there is no obvious indication for this nor is there an 
explanation. 

Navigation is also problematic because the terminology is inconsistent. For 
example, the Ready Reference sources are listed on the home page. At the Ready 
reference resources page, the bottom internal link points to the top of the page and an 
internal link to the Table of Contents. However, the Table of Contents is very near the 
top of the page; it appears the links are redundant. 

The site is also graphically inconsistent. Most pages have the same white 
background, but some in the MU Virtual Subject Library have a brownish texture 
background. Plus, the menu bar (with internal links to main pages on the site) is not 
applied evenly throughout the site. On a positive note, there are descriptive abstracts - 
and commentaries about the different web reference sites. 

Morton Grove’s site contains all of the library’s policies, which can be useful not 
only for patrons, but for other librarians who must devise policies. A policy for selecting 
Web resources is an excellent idea (and a rarity). It helps to inform users about the 
decisions behind the selection process. 

The names of staff members who contributed to the web site are also included. 
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Also, there is a “Site Map" which gives a visual layout of the web site. Although the site 
is not complex, this visual index of the site can help orient users to find areas that are 
pertinent or perhaps not otherwise noticeable. 

The graphics are used when needed, and are large enough to read but do not 
drown out or overshadow the text. The reference subject list is based on the Dewey 
Decimal Classification Scheme, which has internal links to sites further along the large 
reference page. There is just enough sites to be useful and yet restrained so that users 
do not become overburdened by the sheer size of the virtual reference collection, 
seemingly a common occurrance. Although the reference page could be broken up 
into ten separate pages for each Dewey Classification subject heading, (it currently is 
divided into two sections) this site is easy to navigate and apparently includes the 
necessary links for online ready reference. 

The New Orleans Public Library site appears to be still under construction. For 
example, the link to the “Information & Reference Division” points the user to nothing 
except a page with an “under construction” sign. 

There are too few links for each subject heading to be of much help in 
answering a reference question. However, there is a good listing of local web pages 
that could help tourists or new residents discover the flavor of New Orleans. 

The presentation of the home page, which also lists the reference subject headings, is 
weak and unappealing. Gray backgrounds and a cluttering of small font sizes do not 
make for an easy or pleasurable browsing experience. 

One of the subject headings in the reference section is “Big Rat’s selection of 
miscellaneous reference sites.” This may be a local joke or based on myth, but for an 
outsider this does not enhance a professional image of the web site. 

Most of the reference links annotations are too brief and not very useful. 
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Purdue University's reference annotations appear to be accurate descriptions of 
the links. Even the subdirectories have annotations which instructs users about the 
purpose of each section’s purpose. The pages are very consistent visually, with a 
black background, white text and yellow links. Also, the search engine capability, 
located at the library’s home page, is a convenient feature and should be considered a 
standard feature by all large library web sites. 

Unfortunately, there are no subject links in the virtual reference desk. There is a 
page called the “Subject Reading Rooms" broken down by subcategories such as 
“Humanities, Social Sciences, Natural Sciences ...." and so on, but this area is still 
under construction. The virtual reference desk contains links that would be useful for 
ready reference applications - with sites that include dictionaries, time and date, zip 
codes, maps and travel information and other reference sites. 

Appleton Public Library's reference site is simply one long page, which does 
not make it easy to find the appropriate section easily, although subject heading links 
refer to internal parts of this large page. 

All links on the home page are embedded in paragraphs, which forces the user 
to read. This is not convenient (i.e. requires too much time for users to browse) and 
makes it difficult to scan a page and the gray background is not very distinctive or 
appealing. 

There are ample links to local and state information links which could be useful 
to tourists, taxpayers and new residents. For example, there is a pointer to the 
“Wisconsin Elected Officials Home Page”. In addition, the reference page first lists 
roughly 10 Web search engines before providing its list of reference sites -- this does 
not promote access and once again, inconveniencing the user. 
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The University of California at Santa Barbara site lists its “Reference Services” 
site separate from it’s “Information Resources by Subject.” The “Reference Services” 
page was evaluated. Evaluating the “Information Resources by Subject” pages would 
have been too time consuming for this research paper since there appeared to be 
more than 100 subject specialties listed, each with a lengthy set of links. It would be 
convenient for the “Reference Services” page to include internal referral links to the 
“Information Resources by Subject” pages. 

The menu screen at the top of the subject headings list is useful. After each 
subject heading listed in the “Reference Services” page, there is an option to return to 
the top of the page, which is a nice feature since the links are arranged on one long 
page. However, there are not enough links in the reference page. Plus the links are 
not adequately annotated, which could mean there was not much effort placed in 
evaluating the sites. For example, the sole entry under the “Arts” subject heading is a 
link to “The Seven Wonders of the Ancient World” - which could just as easily be 
placed in under the “History” subject heading. 

Curiously, the “General Works” subject heading does not contain links to 
dictionaries, although there are several such Web sites that are available and free of 
charge. 
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Conclusion 

It was not proven whether reference sites that have been updated more recently 
have fewer cases of no URL found than those that have been updated less recently. 

It’s possible that this is merely a result of the random sampling of links. Another 
possible explanation is that the update information is not accurate, i.e., that librarians 
may forget to change the date after revising the page. Probably the best measurement 
for URL accessibility would involve checking a library’s reference site with an HTML 
checker found on the Web. Not only would this be more scientific, it would not be 
influenced by random sampling error. 

Second, it would seem the university library reference sites utilize graphics 
better than the public library reference sites. Also, there was very little difference in the 
quality of sites, although academic sites generally provided more than enough links 
and public library sites provided too little links. 

Most of the reference sites evaluated consisted of excessively large pages of 
links. Although these sites contained internal links from broad subject headings to the 
actual reference links on the Web, this was still inadequate. It appears libraries are 
constructing reference sites with a linear model or some kind of variation of it. Users 
will not enjoy scrolling through large pages to find relevant links. It gives the 
impression that they could do just as well with a Web search engine, without the 
library’s assistance. Morton Grove’s site, which provides users with a graphical 
representation of the different pages available and summaries of those pages, should 
be adopted by other reference sites. Also, Purdue University's search engine feature 
enables users to access the site without getting lost in a maze of hierarchical page 
structure. Yet, users could still get lost sifting through the search engine’s results. Not 
only are abstracts a time saving feature for web searchers because they determine if 
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the site would be helpful or not, but also indicates the webmaster or web team exerted 
effort into the deciding what resources are potentially best for users' information 
needs. Online reference and Web collection policies not only help users understand 
librarians’ thought processes, but potentially adds to the collective information 
available to the library science field. 
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